When a two dimensional first image of one or more physical objects is superimposed on a two dimensional second image of a locale to form a two dimensional third image, the third image is an augmented image. For said third image to be a photorealistic augmented image, when the content of the first image appears to be situated in the third image in a manner that replicates the appearance of a hypothetical fourth image, where said hypothetical fourth image is of a physical scene comprised of all of the physical objects pictured in the first image and all of the physical objects pictured in the second image, with all the objects in the third image and hypothetical fourth image appearing to be positioned in an identical fashion. A plurality of first images may be used to augment a plurality of second images to form a plurality of third images. Each said photorealistic third image has only one corresponding said hypothetical fourth image.
In commercial use of photorealistic image augmentation, said first image is often of one or more objects and said second image is often of a locale. Said augmented third image is then used to evaluate the utility, functionality, aesthetics, or appearance of the object or objects of the first image within the locale of the second image or of the utility, functionality, aesthetics, or appearance of the locale of the second image when containing the object or objects of the first image.
The photorealistic quality of an augmented image depends on how well the photographic properties of said first image match the photographic properties of said second image. Photographic properties include: viewpoint, placement, resolution and scale, lighting and white balance, camera orientation, and the camera's optics.
The viewpoint defines the position and orientation of a camera relative to the subject matter of an image. A requirement for photorealistic image augmentation is that the first image's viewpoint matches the second image's viewpoint.
Digital cameras have an electronic sensor that converts an optical image formed by a camera's optics into an array of pixels to form a digital image which can be stored as an electronic file and manipulated by a computer. In the array of pixels, each pixel has a value that designates a chromaticity, for color images, or a monochrome intensity, for gray scale images. The density of the electronic sensor's array in conjunction with the size of the optical image determines the digital image's resolution and the digital image's scale relative to the physical dimensions of the object. A digital image's resolution, and hence also scale, can be altered by software programs running on a computer. Photorealistic image augmentation depends on the accuracy with which the first image's resolution and image scale matches the second image's resolution and scale at the physical position within the hypothetical fourth image that is occupied by the object of said first image.
Lighting influences the colorimetric appearance of an image as well as the distribution of highlights and shadows within a scene. A camera's white balance setting is used to normalize the colorimetric effects produced by a light source's spectral properties. Post exposure, image editing software can be used to adjust an image's colorimetric properties to approximate the effect of a camera's white balance adjustment. The distribution of shadows in an image is determined by the location of the light source(s) relative to the physical objects in the scene. Photorealistic image augmentation depends on the accuracy with which the first image's light source's(s') properties and camera's white balance settings match the second image's light source's(s') properties and camera's white balance settings.
Photorealistic image augmentation depends on the accuracy with which the first image's vanishing points match the second image's vanishing points.
Skew, pitch, yaw, and rotation differences between the camera orientation used to obtain the first image and the camera orientation used to obtain the second image, will adversely affect the photorealistic appearance of the augmented third image and cause the third image to deviate from the hypothetical fourth image.
In many instances in which photorealistic image augmentation is of commercial value, the first image and the second image are not acquired: by the same person, with the same camera equipment, with the same camera settings, or under the same lighting conditions. Thus, to achieve photorealistic image augmentation, methods are needed to either quantify the salient photographic parameters of the preexisting first image and use said photographic parameters to define the acquisition of the second image or quantify the salient photographic parameters of the preexisting second image and use said photographic parameters to define the acquisition of the first image. Or, to use said photographic parameters to use computer means to adjust the properties of said first image or said second image, after the acquisition of the picture, such that the salient properties are made to more closely match.
Three dimensional computer aided design (3DCAD) models of physical objects are called avatars. Avatars used in conjunction with 3DCAD software running on a computer provide a means of envisioning objects and the arrangement of objects within a specified space. The creation of avatars is time consuming and requires specialized computational tools, and may not portray the object or the setting in a photorealistic fashion. Attempts to blend 3DCAD models with two dimensional photographs of environments in order to reduce the time and skill associated with building 3DCAD environments within which avatars are viewed have limitations. Thus, there is need for an alternative means for readily augmenting an image of a space with an image of an object.
There is also a derived need for bridging the gap between two dimensional photography and three dimensional virtual modeling so that two dimensional views of 3DCAD objects can be used to augment two dimensional images of physical space and photographs of physical objects can be merged into 3DCAD views.
Accordingly, means are needed for quantitatively matching the photographic properties of a predetermined image to the photographic properties of a newly acquired image.